Step 1: Create a Log Stream target endpoint

To create a new target endpoint:

1. In the Manage Endpoint Connections window, select New Endpoint Connection. For information on how to add an endpoint, edit and view its configuration, see Defining and managing endpoints.

2. In the General tab, specify the following information:

   • Name for the endpoint. For example, Log Stream Staging Target.
   • Optionally, provide a description.
   • Select the Target role option.
   • From the Type drop-down list, select Log Stream.

   • In the Storage path field, specify the full path in which to store the streamed data (i.e. the binary file with the record changes). You cannot specify a relative path.

     Warning noteMake sure the staging folder is located on the Replicate Server machine. Qlik recommends using a block device (physical, virtual or iSCSI-based) for storing the streamed data. Using NFS or SMB-based storage is not supported due to the associated latency which could greatly degrade the data transfer performance, as well as due to reduced reliability and compatibility issues.
   • Batch: Specify when to close the batch of operations that will be applied to the target endpoint:

     • Apply batch after (seconds): Length of time that the batch should run. Default is 5 seconds. Maximum permitted time is 60 seconds.

       This parameter effects the start time of the Replication task; until at least one batch is written by the Log Stream Staging task, the Replication task will not start.

     • Apply batch when number of changes reaches: Maximum number of change operations to include in the batch. Default is 10,000. Maximum permitted number of operations is 1,000,000.
   • Retention: Specify when the staging file should be deleted (note that active files will not be deleted):
     • Delete staging files after (hours): Select the check box and specify the maximum time before a file is deleted. Default is 48 hours. Maximum permitted time is 10,000 hours.

     • Delete oldest files when the total size of all staging files exceeds (MB): The maximum size that you want to allocate for the staging folder. If the specified size is reached, Replicate will start deleting files from the oldest to the newest until the total size falls below the upper limit.

       Default is 100,000 MB. The minimum size should be at lease twice the defined rollover size. Maximum permitted size is 1000 TB.

   • Rollover: Specify when to start writing to a new staging file:
     • Roll over file after (minutes): Number of minutes after which a new staging file should be started. Default is 120 minutes. Maximum permitted time is 10,080 minutes (one week).
     • Roll over files larger than (MB): Size of the file after which a new staging file should be started. Default is 500 MB. Maximum permitted size is 100,000 MB.
   • Compression: Optionally move the slider to change the file compression level. The higher the compression level, the greater the processing latency will be. Therefore, a lower compression level (e.g. 2) is recommended when processing speed is more important than saving disk space.
3. Click Test Connection to verify the information you entered and the availability of the target database.
4. Click Save.